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WRISTWATCH TYPE DEVICE 

AND METHOD FOR MOVING 

POINTER 



ickground of Invention 

Field of the Invention 

The present invention relates to a device suitable for use with a portable 
information input device. More particularly, to a wristwatch type device and a method 
for moving a pointer. 

Background Art 

As is well known in the art, owing to recent miniaturization of information 
apparatuses, so-called palm-sized (palmtop type) information apparatuses have 
appeared as well as wristwatch type information apparatuses. 

These information apparatuses do not provide for a keyboard for their 
miniaturization, instead some of them provide for a pointing device in the form of 
button, ball, lever, etc., which are operable in two, four or all directions for performing 
various operations, such as moving a pointer or a cursor on a display screen, scrolling 
a screen, and inputting characters. Alternatively, instead of these pointing devices, 
some of information apparatuses provide for a so-called touch panel, which is 
operated by touching the display screen. 

However, when above-mentioned information apparatuses are provided with a 
pointing device, there is needed at least an operative member such as a button, ball 
and lever; a mechanism for moving this operative member; and a mechanism for 
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detecting an operation of the operative, member, all of which not only prohibit 



miniaturization and water proofing property, but also influence and restrain the 
design for an accessory, especially in the case of wristwatch type information 



apparatuses. 



[0007] 



Assuming that the display itself of the device is of a touch panel type, an 



information apparatus with a certain degree of size is operable by touching with a 
touch pen or a finger, however, for a wristwatch type information apparatus, in 
particular, it is unrealistic to always carry a touch pen with a watch, thus it is to be 
touched by a finger. However, a wristwatch type information apparatus is hard to 
operate due to its small display screen (touch panel), which results in a big problem in 
operability, such as a portion that is touched by a finger is hard to see. Also, when 
providing for a touch panel on the display screen, not only does the transmittance of a 
display screen degrades, but also a fingerprint and stain are likely to be attached, 
which can further degrade the visibility. 

[0008] In addition, for these small information apparatuses, there is always a need to 
restrain power consumption to extend the usable time. 

[0009] The present invention solves the technical problems of the prior art. Therefore, an 
object of the present invention is to provide a device that does not prohibit 
miniaturization and that allows for the adding of water proofing properties. 

[001 0] Another object of the present invention is to provide a device that restrain power 
consumption to extend the usable time. 



Summary of Invention 



[0011] 



A feature of the present invention includes a device having a display controller for 
controlling a display position of the pointer on the display screen, a displacement 
detector for detecting a displacement of the device itself, and a pointer mover for 
moving the pointer on the display screen based on the detected displacement of the 



device itself. 



[0012] 



In another aspect of the present invention, a wristwatch type device of the present 
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invention comprises a touch sensor mounted in a case or an attached belt for 
performing a predetermined operation on an object displayed on the screen. 

[001 3] In a further aspect of the present invention, a method for controlling a device 
comprises a first step of taking an image of a physical object facing the device 
continuously and detecting a relative displacement between the taken object and a 
display. The method includes a second step for changing a display position of a 
pointer displayed on the display based on the detected displacement. 

[0014] In a still further aspect of the present invention, there is provided a method for 
moving a pointer displayed in a display of a device. The method comprises the steps 
of: detecting a displacement of the device when moving the device; and changing a 
^3 display position of the pointer displayed in the display based on the detected 

If] displacement. 

: " i 

^ [001 5] Various other objects, features, and attendant advantages of the present invention 

13 

ill will become more fully appreciated as the same becomes better understood when 

I|Ba considered in conjunction with the accompanying drawings, in which like reference 

□ characters designate the same or similar parts throughout the several views. 

^ Brief Description of Drawings 

:~ [001 6] Figs. 1 A and 1 B show the illustration of a wristwatch type device according to the 
present invention, wherein Fig. 1 A is the elevation view and Fig. 1 B shows how to 
operate the wristwatch type device attached to the wrist. 

[001 7] Fig. 2 shows a schematic of the wristwatch type device. 

[001 8] Fig. 3 is a flowchart of a process for moving a pointer in a first embodiment of the 
present invention. 

[001 9] Fig. 4 shows an image taken in by the image sensor. 

[0020] 

Fig. 5 shows a flow of the image processing of the image taken in by the image 
sensor, wherein (a) shows a reference image area and a search area for the first 
image; (b) shows how to scan the object image area within the search area for the 
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second image; (c) shows a position of the object image area corresponding to the 
reference image area. 

[0021] Fig. 6 shows how a cursor moves in the second embodiment of the present 
invention. 

[0022] Fig. 7 is a flowchart of a process for moving a cursor in the second embodiment of 
the present invention. 

[0023] Figs. 8A and 8B are diagrams showing how to operate a wristwatch type device, 
wherein Fig. 8A shows moving the device upward and Fig. 8B shows moving the 
device downward subsequently. 

i3 [0024] Figs. 9A through 9D show moving patterns arranged in time-series from (a) to (d) 
ill that are taken in by moving the wristwatch type device as shown in Fig. 8. 

is 

|j [0025] Fig. 1 0 is a sectional side elevation of the wristwatch type device according to the 

? J 

iy other embodiment of the present invention. 



13 
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[0026] Fig. 1 1 shows a still further example of the wristwatch type device according to 
the present invention. 

Detailed Description 

[0027] For the present invention, when moving the device itself equipped with the display 
screen, a displacement of the device is detected, thereby moving the pointer on the 
display screen. In this way, it becomes possible to operate a device, without using a 
mechanical pointing device or a touch panel type display screen as before. 

[0028] The displacement detector may comprise an image sensor, wherein an image 

sensed by the image sensor is processed to obtain a displacement of the device itself. 
The image sensor preferably includes a complementary metal-oxide semiconductor 
(so-called CMOS), a charge coupled device (so-called CCD), or an infrared sensor. 



[0029] 



The device may further comprise an operator activating the image sensor, wherein 
the image sensor is turned off while not operating the pointer, thereby restraining the 
power consumption. Furthermore, the operator may comprise a function for directing 
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a selection of an object pointed by the pointer or an execution of predetermined 
processing defined for the object, whereby the operator have a plurality of functions, 
thereby keeping the number of operator to a minimum. 

[0030] In this way, providing for a touch sensor around a display in a wristwatch type 
device with a small display, an operation can be performed without touching the 
display, thereby being superior in operability. Moreover, a device can be miniaturized 
by employing not a mechanical pointing device but a touch sensor. 

[0031] Further, providing for touch sensors on both sides of the display, the hold 
characteristic of the device during operation increases. 

[0032] The wristwatch type device may further comprise displacement detector for 
;Q detecting a displacement of the display; and pointer position changing means for 

:fj changing a display position of a pointer based on the detected results, thereby 

iSJ 

!y moving the pointer displayed on the screen. 

13 

~ [0033] A first step according to the present invention further comprises the steps of: 
calculating a motion vector at a certain place in an image that was taken multiple 
times; and obtaining a relative displacement between the object and the display based 
on the calculated motion vector. A motion vector in a certain place is calculated based 
on a position of the certain place in a principal image (e.g., the first image) and a 
position of a place corresponding to the certain place in another image (e.g., the 
second image) that was taken apart in time from the principal image. That is, it is 
obtained where an image at a certain place in the first image moved in the second 
image. 

[0034] In this case, when moving the device relative to the object, the relative 

displacement between the object and the display is obtained by inverting a sign of the 
motion vector. 

[0035] Alternatively, the first step may comprise the steps of: generating a time-series 
moving pattern of a certain place based on a position of the certain place in a 
principal image and a position of a place corresponding to the certain place in a 
plurality of other images; and comparing the generated time-series moving pattern 



U 



Page 5 of 36 



with a plurality of model patterns to select a most approximate model pattern. The 
second step may comprise the step of changing a display position of the pointer 
based on a moving pattern that was defined for the selected model pattern. 

[0036] The method may further comprise the steps of starting to detect a displacement 
of the device when a predetermined startup operation is performed by a user. With 
this configuration, when not detecting a displacement of the device, a sensor for 
sensing a displacement is kept off-state, thereby restraining power consumption. 

[0037] Now the present invention will be described based on three embodiments shown 
in the attached drawings. 

[0038] Fig. 1 is a diagram for illustrating a wristwatch type device according to the 
present invention. In Fig. 1 , a symbol 1 0 is a wristwatch type device, 1 1 is a case 
supporting the body of the wristwatch type device 1 0, 1 2 is a display comprising a 
display screen that is composed of an LCD panel, for example, provided on the front 
side of the case 1 1, and 1 3A and 1 3B are an attached belt of a wristwatch type device. 

[0039] As shown in Fig. 1(a), there is provided an image sensor 14 in a part of the display 
1 2, which serves as a displacement detection means or a displacement detecting 
section. The image sensor 14 takes an image that faces the display 12 on the display 
side, in other words, an image when viewing outside (of the wristwatch type device 
1 0) from the display side 1 2. That is, such as a user's face is taken as an imaging 
object when he or she looks inside the display 1 2 with facing the wristwatch type 
device 1 0. 

[0040] The image sensor 1 4 is preferably a CMOS or CCD. In the embodiment of the 

present invention, a CMOS of 36 X 36 dots or so is employed as an image sensor 14 
for the reason that the required minimum resolution is assured and power 
consumption is low. 

[0041] On the front side of the case 1 1 , there are provided, as the operative means, a 

pair of two touch sensors 1 5A and 1 5B around the display 12, for example upper side 
and lower side. 
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[0042] As shown in Fig. 2, there is provided a control section (display control means) 20 
inside the case 1 1 for controlling the contents displayed in the display 12. The control 
section 20 comprises a storage section 21 for storing predetermined programs, a 
processing section (pointer moving device, pointer position changing device) 22 for 
processing the contents displayed in the display 12 based on the programs stored in 
the storage section 21 , an image processing section 23 for performing predetermined 
processing based on images that are taken by the image sensor 14, a switch section 
24 for turning the image sensor 14 on-state, and a detection section 25A and 25B for 
detecting a user touching the touch sensor 1 5A and 1 SB. 

The processing section 22 issues a command to activate the image sensor 14 
using the switch section 24 when detecting either the touch sensor 1 5A or 1 5B is 
being touched in the detection section 25A or 25B. Further, the processing section 22 
causes the image processing section 23 to perform predetermined image processing 
on an image taken by the image sensor 14, when detecting both touch sensors 1 5A 
and 1 5B are being touched in the detection sections 25A and 25B while the image 
sensor 14 is on-state. Then, the movement of the pointer P shown in Fig. 1 is 
processed based on the results, thereby changing the display position of the pointer P 
on the display screen of the display 12. In addition, as shown in Fig. 1, when detecting 
either the touch sensor 1 5A or 1 SB is tapped while the pointer P indicates any icon (II 
to 14) on the display screen, the processing section 22 determines the click operation 
has occurred and performs predetermined processing based on the programs stored 
in the storage section 21. 

[0044] As shown in Fig. 1 (b), such a wristwatch type device 1 0 is attached to the wrist of 
a user by the attached belt 1 3A and 1 3B. In the display 1 2 of the wristwatch type 
device 1 0, predetermined display contents are displayed based on the programs 
stored in the storage section 21 . As with personal computers and various portable 
information terminals, icons and menus are displayed in the display screen of the 
display 1 2. 

[0045] 

In the present embodiment, as shown in Fig. 1 (a) for example, a plurality of icons 
II to 14 are displayed as an object for the initial screen in the display screen of the 
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display 1 2. Also, the pointer P is displayed in the display 1 2 for performing an 
operation. 

[0046] For a wristwatch type device 10, just like a pointer is operated by mouse in a 

typical personal computer, a pointer P is moved in the display screen of the display 1 2 
to perform various operations. For example, indicating any icon (e.g., II) with a 
pointer P and performing a predetermined operation (e.g., double-click) in this 
condition, predetermined processing is performed which is defined for an icon II . 

[0047] For the wristwatch type device 1 0 according to the present embodiment, a pointer 
P is moved within the display screen of the display 12 by simply moving the 
wristwatch type device 1 0, without the need for a pointing device such as a button, 
13 ball, lever, and let alone a mouse and touch panel. When wearing a wristwatch type 

If! device 1 0 on the wrist, one may simply move his or her arm. 

1 

ill [0048] Usually, there is some allowance for an attached belt when wearing a wristwatch 

13 

\%l type device on a wrist, thus it is possible to move only a watch on an arm (wrist) 

without moving an arm. Therefore, as shown in Fig. 1(b), according to the present 
13 embodiment, it is good enough to grasp this wristwatch type device 1 0 with an 

= ^ opposite hand to the one wearing this device and move it on his arm in the direction 

M shown by the arrow in the drawing. 

E ~~ [0049] At this time, a displacement of the wristwatch type device 1 0 is detected based on 
a motion vector of an imaging object (e.g., a head of a user who is looking inside the 
display 12) within an imaging range of the image sensor 14 mounted in the display 
1 2, then the pointer displayed in the display 1 2 is moved based on this displacement. 
Now the detailed flow will be described in reference to a flowchart of Fig. 3. 

[0050] As for the processing in the wristwatch type device 10, while the image sensor 14 
is off-state (step SI 00), the detection section 25A or 25B detects that a user touches 
either the touch sensor 1 5A or 1 5B with his finger or the like according to a 
predetermined startup operation (step SI 01), then the image sensor 1 4 is turned on 
(step SI 02). 



[0051] 



After the image sensor 14 was turned on, the detection sections 25A and 25B 
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detect whether the touch sensor 1 5A or 1 5B is touched within a predetermined time 
(measured by a timer) (step SI 03). If not, the image sensor 1 4 is turned off and 
returns to the wait condition (step 1 04). 

[0052] If detecting that the touch sensor 1 5A or 1 5B is touched within the predetermined 
time, it is detected whether both touch sensors are touched. If both of them are not 
touched at the same time, the after-mentioned step (SI 09) is continued (step SI 05). 

[0053] If detecting that both touch sensors 1 5A and 1 5B are touched, an optical flow is 
generated in the image processing section 23 based on the image taken by the image 
sensor 14 (step SI 06). 

[0054] In order to generate the optical flow, the image processing section 23 takes in an 
image taken by the image sensor 14 every predetermined time (e.g., 32 msec). As 
shown in Fig. 4, a plurality of reference image areas (RA) are set in the taken image (3 
x 3 = 9, in the example shown in Fig. 4), wherein the motion vector (an arrow shown 
in Fig. 4) is calculated for each reference image area RA between its corresponding 
image at a predetermined time later. 

[0055] Fig. 5 is a diagram for illustrating how to calculate a motion vector for each 

reference image area RA, wherein a single reference image area RA is set for an image 
for simplifying the explanation. 



[0056] 



As shown in Fig. 5(a), a reference image area RA is set at a predetermined position 
for a principal image Pt that was taken at time t , then a search area SA, which is 
greater than the reference image area RA, is set for this reference image area RA. This 
search area SA is set for the display screen of the display 1 2, that is so to speak fixed. 
Then, for an image Pt ^ taken at time t ^ after a predetermined time passed, an object 
image area OA, which has the same size as the reference image area RA, is scanned in 
the search area SA as shown by an arrow in Fig. 5(b), and then a position is searched 
which has a minimum difference from the reference image T in the reference image 
area RA of the principal image Pt . At this point, the following equation is used for 
obtaining a difference between the reference image T in the reference image area RA 
of the principal image Pt and the object image I in the object image area OA of the 
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image Pt ^ . 
[0057] Equation 1 

R=22 I I (i, j)-T(I, j) 

i=l j=l 



m 
ly 
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[0058] In the above equation 1, i is a position of a pixel in X direction (e.g., horizontal 

direction in the drawing) in the object image I and reference image T, j is a position of 
a pixel in Y direction (e.g., vertical direction in the drawing) in the object image I and 
reference image T, M is the number of pixels in X direction in the object image area 
OA and reference image area RA, and N is the number of pixels in Y direction in the 
object image area OA and reference image area RA. 

[0059] As shown in Fig. 5(c), when scanning the object image area OA in the search area 
SA for the image Pt ^ , an object image I corresponding to a position that has a 
minimum residual R calculated by equation 1 is a position corresponding to the 
reference image T of the principal image Pt , where the reference image T is to 

q move. Hereby, the motion vector (an arrow shown in Fig. 5(c)) of the reference image 

:S T during a predetermined time (t ^ to t ) is obtained. 

m 

III [0060] In the same manner as described above, each motion vector (see an arrow in Fig. 

n 

5 r ; 4) is obtained for each reference image area RA shown in Fig. 4. A search area SA is 

=P set individually for each reference image area RA. 



[0061] A distribution of motion vectors in a plurality of reference image areas RA is the 
optical flow obtained in step SI 06. 

[0062] In next step SI 07, an average of each motion vector in the optical flow is 

calculated. Since this average is a motion vector of an image taken by the image 
sensor 1 4 for a predetermined time (t ^ to t Q ), then a sign (plus, minus) of this 
motion vector is inverted, which becomes the motion vector of the wristwatch type 
device 1 0. 

[0063] Then, multiplying the obtained motion vector by a predetermined coefficient to 
transform it to the motion vector of the pointer P in the display screen, a display 
position of the pointer P in the display screen is moved based on this motion vector 
(step SI 08). 

[0064] Next, it is determined whether the moved pointer P overlaps with any icon II to 14 
(step SI 09). If not, the process goes back to step SI 03 to continue the processing for 
moving the pointer P, while if the pointer P overlaps with any icon, the process waits 
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for an operation on either the touch sensor 1 5A or 1 5B for a predetermined time (step 
SI 1 0). It is detected how many times the touch sensor 1 5A or 1 5B have been touched 
in the predetermined time, then depending on the result, if the detected times are 
zero, the process goes back to step SI 03, if the detected times are one, it is 
determined to be a single click operation, wherein an icon that overlaps with the 
pointer P (e.g., II) is selected (step SI 1 1). If the detected times are two, it is 
determined to be a double click operation, wherein a predetermined operation, for 
example, an activation of a related application, which is defined for an icon (e.g., II) 
that overlaps with the pointer P is performed (step SI 12). After that, in step SI 03, the 
process waits for a touch on the touch sensor 1 5A or 1 5B, as a result, if no touch 
occurs for a predetermined time, the image sensor 1 4 is turned off in step SI 04. 

;*J [0065] In this way, moving the wristwatch type device 10 itself, the moving operation of 

I S3 the pointer P on the display screen of the display 1 2 is performed. This dispenses with 

ill 

a mechanical pointing device such as a conventional button type one, which enables 
ill improvement of miniaturization and water proofing property of the wristwatch type 

device 1 0 and further enhances flexibility of design. Moreover, one does not need to 
^ touch the display 1 2 like a touch panel, so it excels in operability and also can avoid 

the display smearing. 



I s8 * 



[0066] In addition, as the touch sensor 1 5A and 1 SB functions as a switch for the image 
sensor 1 4, as long as one does not touch the touch sensor 1 5A and 1 SB, the image 
sensor 14 does not operate, which restrains power consumption and extends a 
consecutive usable time. Moreover, the touch sensor 1 5A and 1 5B are not mounted in 
the display 12 but around its periphery, which prevents the display 12 smearing. 

[0067] Furthermore, while both the touch sensors 1 5A and 1 SB are touched, the 

detection of movement is performed in the image sensor 14. Thus, if one touches the 
touch sensors 1 5A and 1 5B, which are disposed on both the top and bottom sides of 
the display 12, with a thumb and forefinger, the case 1 1 of the wristwatch type device 
10 is held stable, thus its movement is performed easily and reliably. 

[0068] Next, a second embodiment of the present invention will be described for a 

device, a wristwatch type device, a method for controlling the device, and a method 
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for moving a pointer. While in the above first embodiment, a pointer P is moved to any 
position on the display screen of the display 12 by moving the wristwatch type device 
1 0, in the second embodiment described below, movement of a cursor is determined 
based on a moving pattern of the wristwatch type device. In the following description, 
common components with the above first embodiments are shown by the same 
reference numbers and are omitted about their explanation. As shown in Fig. 6, a 
wristwatch type device 1 0' according to this embodiment is completely the same as 
the wristwatch type device 1 0 described in the first embodiment in terms of device 
configuration, wherein only the difference is the processing in the control section 20 
shown in Fig. 2, that is, the programs stored in the storage section 21 . 

[0069] As shown in Fig. 6, for this wristwatch type device 1 0', icons II to 14 are displayed 

=y in the display screen of the display 1 2 as an initial screen. In this condition, a cursor C 

=n 

J« as a pointer is displayed by inverting any one of icons II to 14 (e.g., icon II) between 

\& 

IU black and white. 

13 

III 

p [0070] As shown in the flowchart of Fig. 7, for the wristwatch type device 1 0', the image 

5 sensor 14 is off-state in the initial condition (step S200). When the detection section 

Q 25A or 25B detects that a user touches either the touch sensor 1 5A or 1 5B with a 
finger or the like (step S201 ), then the image sensor 1 4 is turned on (step S202). 



n 



[0071 ] After the image sensor 1 4 was turned on, if the touch sensor 1 5A or 1 5B is not 
touched within a predetermined time (step S203), the image sensor 1 4 is turned off 
and returns to the initial condition (step 204). If the touch sensor 1 5A or 1 5B is 
touched within the predetermined time, it is detected whether both touch sensors 1 5A 
and 1 5B are touched (step S205). If both of them are not touched at the same time, 
the after-mentioned step (S210) is continued. 

[0072] If detecting that both touch sensors 1 5A and 1 5B are touched, just like the first 
embodiment, an optical flow is generated in the image processing section 23 based 
on the image taken by the image sensor 14, and further an average of each motion 
vector of the optical flow is calculated (step S206). Note that the optical flow is 
generated within a predetermined time (e.g., 160 msec) in time-series from an image 
taken by the image sensor 1 4 every predetermined time (e.g., 32 msec). 
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[0073] Next in step S207, an input pattern of the wristwatch type device 10' is generated 
from the average of the above-mentioned motion vector. 

[0074] Assuming that the wristwatch type device 10' is moved upward first, then 

downward within a predetermined time in step S206, as shown in Fig. 8(a) and Fig. 8 
(b). 

[0075] The average of the motion vector of the optical flow obtained for the 

predetermined time in step S206 is shown in Fig. 9(a) to (d). Fig. 9 shows images that 

are taken in by the image sensor 14 every predetermined time (32 msec), which are 

arranged in time-series from (a) to (d). Fig. 9(a) is an image F that was taken in by 

the image sensor 1 4 for the second time, (b) is an image F taken in for the third 

time, (c) is an image F taken in for the fourth time, (d) is an image F taken in for 

4 5 

the fifth time. In each drawing, a dotted line shows a position of an image that was 
|fl taken in the last time, wherein the difference from that position is to be an average of 



53 i 



the motion vector V , V ^ , V ' and V . For example, for an image F , a position 



shown by the dotted line corresponds to a position of a principal image that was taken 
in for the first time, wherein the difference from a position of an image shown by a 

n 

"\ solid line becomes an average of the motion vector V . 

"M 0 

U [0076] In this way, a time-series moving pattern A that is a series of data of averages of 

s -3 

j*[ motion vectors V ' , V ^ , V and V is represented as follows. 

[0077] A = (V o' V i' V 2' V 3 ) 

[0078] On the other hand, a plurality of model patterns are stored in advance in the 

storage section 21. This model pattern is data that models the assumable movement 
of the wristwatch type device 1 0 f when operating the cursor C. Each pattern is defined 
as a moving pattern either with cursor movement in a predetermined direction (e.g., 
total of four directions including up, down, left and right) or without cursor 
movement. Fig. 6 depicts how the cursor C displayed in the display 1 2 moves in total 
of four directions between icons II to 14. For example, if the cursor C is a moving 
pattern that moves to the right, the cursor C moves from the icon II shown in the 
upper left of Fig. 6 to the icon 13 shown in the upper right. 
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[0079] A plurality of model patterns registered like this are matched with the time-series 
moving pattern (input pattern) A obtained in step S207, thereby recognizing a model 
pattern that most approximates to the pattern A among a plurality of model patterns. 

[0080] Next in step S209, according to a moving pattern defined for the recognized 

model pattern, the cursor C in the display 1 2 is moved (if approximating to the model 
pattern without cursor movement, the cursor C does not move). For example, in an 
example of the time-series moving pattern A shown in Fig. 9, the pattern A is 
recognized to approximate to a model pattern when moving the cursor C upward, 
thus the cursor C is moved upward according to the moving pattern (i.e., upward 
movement) defined for this model pattern. In this way, based on the time-series 
moving pattern A when moving the wristwatch type device 1 0', the cursor C is moved 

13 

,p to the left, right, upward and downward, as shown in Fig. 6. 

m 

|-p [0081] After the cursor C is moved like this (sometimes it does not move), in step S21 0, it 
□ is detected how many times the touch sensor 1 5A or 1 5B have been touched in the 

j ^ predetermined time. If the detected times are zero, the process goes back to step 

S203, if detected times are two, it is determined to be a double click operation, 
wherein a predetermined operation, for example, an activation of a related application 
that is defined for an icon (e.g., II) that overlaps with the cursor C is performed (step 
S211). 

[0082] As mentioned above, for the wristwatch type device 1 0', the cursor C is moved on 
the display screen of the display 12 by moving the wristwatch type device 10' itself, as 
with the above-mentioned first embodiment. This enhances flexibility of design as 
well as brings about the same effects as the first embodiment, including 
miniaturization of the wristwatch type device 10\ improvement of water proofing 
property and operability, and stain proofing property of the display 12. 

[0083] By the way, in the first embodiment, the pointer P might be displayed in a . 

different position than icons II to 14, whereas in the second embodiment, the cursor C 
moves between icons II to 14 (sometimes does not move) to always indicate any one, 
thus the cursor C is easily operated to move. 
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[0084] It is noted that in the above second embodiment, the moving pattern for the 

cursor C includes total of four directions including up, down, left and right, however, 
it should not be limited to this. For example, an oblique direction may be added or 
only upward and downward directions or only right and left directions may be 
considered depending on a display of icons or an operation menu in the display 1 2. 
For a matching technique for matching a time-series moving pattern A with a model 
pattern or a generation technique for generating a time-series moving pattern A, any 
technique may be employed as long as a required object is achieved that a moving 
direction of a cursor C be defined as a predetermined direction when moving a 
wristwatch type device 1 0'. Further, a cursor C is moved in the above second 
embodiment, however, a pointer P may be moved by a similar technique. 

[0085] In the above first and second embodiments, an image sensor 14 is employed to 

detect movement of the wristwatch type devices 10 and 10 f and an image taken by the 
image sensor 14 is used as the basis, however, an infrared sensor, for example, may 
be employed as an image sensor 14 according to the purport of taking an image. An 
infrared sensor can take an image even in a dark environment. 

[0086] Moreover, this image sensor 14 was provided in the display 1 2, however, it may 

be attached to other places such as the case 1 1 or attached belts 1 3A and 1 3B as long 
as an image can be taken for an imaging object. 

[0087] Also, a heat-sensitive sensor can be used instead of an image sensor 14. The 
heat-sensitive sensor recognizes a user's face looking inside the display 1 2 as a 
heating element whose temperature is higher than the surrounding area, and detects 
movement of the heating element in the taken image when moving the wristwatch 
type devices 1 0 and 1 0'. A configuration similar to the above first and second 
embodiments could be implemented in this manner, as well. 

[0088] Next, a third embodiment of the present invention will be described, a wristwatch 
type device, a method for controlling the device, and a method for moving a pointer. 
While in the above first and second embodiments, an image sensor 14 is employed to 
detect the movement of the wristwatch type devices 10 and 10', the third embodiment 
employs a strain sensor instead. 
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[0089] Fig. 1 0 is a sectional side elevation of the wristwatch type device 50 according to 
the embodiment, wherein this wristwatch type device 50 includes the body 51 with 
the display 12 provided on its surface, a base 52 that supports the body 51 with 
freedom within a plain along the display 12, a strain sensor (displacement detection 
means) 53 provided between the backside of the body 51 and the base 52, and 
attached belts 54A and 54B. A peripheral wall 52a upstanding toward the body 51 is 
formed on the periphery of the base 52, which regulates the movement of the body 
51 relative to the base 52 within a predetermined distance. 

[0090] This wristwatch type device 50 provides for a strain sensor 53 instead of an image 
sensor 14 of the wristwatch type devices 10 and 10' shown in the above first and 
second embodiments, wherein the strain sensor 53 senses the movement of the body 
51 that can be moved relative to the base 52, and wherein a pointer P (see Fig. 1) or a 
cursor C (see Fig. 6) that are displayed in the display 1 2 is moved based on this 
movement. A method for moving the pointer P or the cursor C based on the sensed 
movement of the body 51 is not described here since the same method as the above 
first or second embodiment may be employed. 



[0091] Alternatively, the strain sensor 53 may sense the vertical strain between the body 
I 555 51 and the base 52 (i.e., the direction connecting the body 51 and the base 52) to 

fq recognize it as a click operation. 

[0092] Such a wristwatch type device 50 may be attached to a user's wrist by an attached 
belt 54A and 54B. This fixes the base 52 to the user's wrist. When the user holds the 
body 51 with his fingers and moves the body 51 in a direction he wants to move the 
pointer P or the cursor C, if the body 51 provides for the touch sensor 1 5A and 1 5B 
(see Fig. 1) in the same positions as in the above first and second embodiments, the 
same operation method is applied as in the case of wristwatch type devices 10 and 
1 0'. When the user moves the body 51 , a relative displacement occurs between the 
body 51 and the base 52, which is sensed by the strain sensor 53. A motion vector of 
the body 51 is obtained based on the sensed displacement, whereby the pointer P and 
the cursor C are moved according to the motion vector. 



[0093] 



As for the above-mentioned wristwatch type device 50, the same effects are 
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achieved as in the above first and second embodiments. 

[0094] While the strain sensor 53 is provided between the body 51 and the base 52 in the 
third embodiment, the base 52 may be removed in another configuration. That is, the 
strain sensor 53 is provided on the backside of the wristwatch type device 50, wherein 
the strain sensor 53 touches a user's wrist when he or she wears the wristwatch type 
device 50. According to this configuration, when a user moves the wristwatch type 
device 50, a relative displacement between the wrist and the body 51 is sensed by the 
strain sensor 53, thereby achieving the same effects as in the above third 
embodiment. It goes without saying that the attached belts 54A and 54B are attached 
to the body 51 . 

13 [0095] Alternatively, an acceleration sensor may be built in the wristwatch type device 
Jjjj according to the purport that the wristwatch type device be moved to move the 

pointer P or the cursor C where the movement is detected. 



Li 



[0096] Further, in the sight of operation without touching the display device 12, touch 
sensors 63A and 63B may be provided around the display 1 2 on the attached belts 
62A and 62B, which are attached to the upper and lower sides of the body 61 of the 
wristwatch type device 60, as shown in Fig. 1 1 . In this case, the pointer P (see Fig. 1) 
or the cursor C (see Fig. 6) that is displayed in the display 1 2 is moved by these touch 
sensors 63A and 63B. 

[0097] For example, the touch sensor 63A senses the movement of horizontal direction 
of the touched finger, while the touch sensor 63B senses the movement of vertical 
direction of the touched finger, wherein the pointer P and the cursor C are moved 
based on the sensed movements. 

[0098] In such a wristwatch type device 60, an operation is performed using touch 

sensors 63A and 63B that are provided on the upper and lower sides of the display 
12, thus the display 12 does not smear. In addition, as mechanical operative members 
are unnecessary, sufficient effects are brought about in the improvement of 
miniaturization and water proofing property as well as flexibility of design. 



[0099] 



It is also possible to unite touch sensors 63A and 63 B with the body and to attach 
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the attached belts (not shown) there, whereby a user can change the attached belt to 
his or her taste. 

[0100] While in the above embodiments icons II to 14 are displayed in the display 12, the 
number and location of icons as well as the kind and contents of applications 
activated are no object. Further, an operation menu or the like may be displayed as a 
character string instead of icons II to 14. Furthermore, when inputting characters in 
the display 1 2, for example, it is also possible to move a cursor that indicates the 
input position, using the above-mentioned configuration. 

[0101] Further, while touch sensors 1 5A and 1 5B are provided to perform various 

operations in the above each embodiment, their installation locations and the quantity 
are variable and further alternative switches may be employed instead of touch 
sensors 1 5A and 1 5B. In this case, if any other switches that are provided for other 
uses are available, new switches do not need to be provided. 

[0102] While so-called click operation has been performed by touching the touch sensors 
1 5A and 1 5B when the pointer P or the cursor C is located on the icons II to 14, 
another operation method may be employed instead. For example, a predetermined 
operation may be detected by taking it with the image sensor 14. By way of example, 
while a user moves the wristwatch type device 1 0 or 1 0' near and apart from his face, 
this operation could be detected as a click operation from an image taken by the 
image sensor 1 4. 

[01 03] While a device has been shown as a wristwatch type in the above-mentioned 

embodiments, the present invention is applicable to various kinds of apparatuses such 
as a palmtop type portable information terminal, digital camera, cellular phone 
terminal, etc. For a so-called video phone that provides for a camera function among 
cellular phone terminals, the camera may be used as an image sensor. 

[0104] 

The present invention may be applied to both the notebook type and desktop type 
personal computers. In such cases, it is hard to move the personal computer itself, 
thus the image sensor may be mounted in the monitor or the keyboard, wherein a 
pointer P and a cursor C similar to the above embodiments are operable by a user 
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moving his hand in front of the image sensor, which takes an image of its movement. 
A personal computer like this is particularly useful for disabled persons. 

[01 05] As mentioned above, according to the present invention, flexibility of design is 

enhanced as well as various effects are achieved, including miniaturization of a device, 
improvement of water proofing property and operability, and stain proofing property 
of the display screen. In addition, power consumption can be restrained. 

[01 06] It is to be understood that the provided illustrative examples are by no means 
exhaustive of the many possible uses for my invention. 

[01 07] From the foregoing description, one skilled in the art can easily ascertain the 

essential characteristics of this invention and, without departing from the spirit and 
scope thereof, can make various changes and modifications of the invention to adapt 
it to various usages and conditions. 

[01 08] It is to be understood that the present invention is not limited to the sole 

embodiment described above, but encompasses any and all embodiments within the 
scope of the following claims: 
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